-
Notifications
You must be signed in to change notification settings - Fork 5.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Auto-grown sparse table #9897
Auto-grown sparse table #9897
Conversation
@@ -69,5 +116,66 @@ void DeserializeFromStream(std::istream& is, SelectedRows* selected_rows, | |||
TensorFromStream(is, selected_rows->mutable_value(), dev_ctx); | |||
} | |||
|
|||
bool SelectedRows::HasKey(int64_t key) const { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe add some comments here to explain why we need these APIs for SelectedRows
data structure.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done.
} | ||
|
||
bool SelectedRows::Get(int64_t key, framework::Tensor* value, | ||
int64_t row) const { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
row is the same as key?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No, it's the offset of the given value pointer, maybe I can rename it to offset
...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As we discussed, need to refine the Get
function as: vector<int64_t> empty_keys Get(vector<int64_t> keys, SelectedRows<T> result);
const T* old_ptr = | ||
tensor_->memory_size() == 0 ? nullptr : tensor_->data<T>(); | ||
if (old_ptr != nullptr) { | ||
std::copy(old_ptr, old_ptr + tensor_->numel(), ptr); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
better to use memory::copy
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM!
Fixed #9841
Taks: #9211